Unions and wage gains: a DID approach

Background

As you may have seen on my twitter account, I have been working closely, implementing Callaway and Sant'Anna (2021) estimator, as well as learning about the problems and potential solutions regarding DID when panel data is available.

As I go deeper into learning about the topic, and working on a side project that may benefit from this tools, one starts to see many potential applications. One of this being the analysis of union wage gains.

A recent approach that called my attention, in particular for its relatively easy implementation, and potential flexibility, has been the one proposed in parallel by Gardner, John (2021) “Two Stage Difference-in-differences”, Working paper, and by Kirill Borusyak (UCL), Xavier Jaravel (LSE) and Jann Spiess (Stanford) 2021 “Revisiting Event Study Designs: Robust and Efficient Estimation”.

For me, these two papers propose the same methodology, with Borusyak, et al (2021), being the more advance and flexible, (specially in terms of the implementation). Using their implementation (see ssc install did_imputation), I decided to prepare a small example to see how this could be applied for analyzing the gap gains of unions, using an extract from the NLSY.

As you will see, I started very optimistic, finding results that made sense with common wisdom regarding what unions do. But, as you will see, some questions arise on whether or not I'm really capturing a causal effect, or if some other factors in the background are generating the observed effects.

One last word before I walk you thought this excercise. This is meant only as a classroom example of the application of some of the new methodologies on DID. As such, only minimal assumptions on the model specification will be used. So lets start.

Setup

The model

As I mentioned before, the methodology I'll be using follows Gardner(2021) and Borusyak et al (2021). Both are based on a kind of imputation approach that could be summarized as follows:

Assume that a person log wages (y0_{it}), in absence of unions, is a function of only three factors. Individual unobserved but fixed effects ( $\gamma_i$ ), some year effects that are common to all individuals ( $\delta_t$ ), and some idiosyncractic error $e_{it}$ . We can call this the potential outcome in absence of treatment.

$y0_{it} = \gamma_i + \delta_t + e_{it}$

Some people, however, may become unionized. In this case, the unionization can be considered as a treatment variable, which could have a potentially heterogenous effect on individuals. The source or direction of heterogeneity is not known. For simplicity, we can say that potential wages when unioninzed are simply written as:

$y1_{it} = \gamma_i + \delta_t + \theta_{it}+ e_{it}$

Notice that in this case, im assuming the effect is fully heterogeneous, since it could vary by person, time, or a combination of both.

In this setup, the wage gains of unions, at the individual level, could be simply estimated as the difference between both potential outcomes. (when unionized - not unionzed)

$TE_it = y1_{it}- y0_{it} = \theta_{it}$

If we can observe both potential outcomes, we could then estimate average treatment effects (ATE), treatment effects on the treated (ATT), on the untreated (ATU), or even distributional treatment effects (QREG perhaps?).

The question, however, is how to identify this effect.

If we assume that the treatment of unionization is a permanent effect (once a union member, always a union member), one of the ways that have been used to estimate this effect has been using a generalized DID approach. Basically, this implies the estimation of the following model:

$y_{it} = \gamma_i + \delta_t + \theta * union_{it}+ e_{it}$

However, many papers have already discuss the limitations of this model approach, and the potential bias of identifying the treatment effect $\theta$ . The basic idea is that this model estimates a weighted average of the "treatment effect" using the timing differential and differences across unions. However, it does not distinguish from "bad" comparisons where previously treated unions are used as controls for indivduals who are "treated" later. Goodman-Bacon (2021) provides a full explanation of this problem, and I provide my own explanation of this on (find one of the links on the left).

Because $\theta$ is contaminated because of the "bad" comparisons, it stands to reason that the estimation of $\gamma 's$ and $\delta's$ may also be biased. So the Two way fixed effect, may not be what you want to do.

So what is the alternative?

Gardner (2021) and Borusyak, et al (2021) suggests that good estimations for the ATE $\theta$ is still possible if one could estimate, impute, or extrapolate potential outcomes for the treated units, by using data for those observations that were not treated yet.

This is a very intuitive approach. It basically suggests to estimate the following model:

$y_{it} = \gamma_i + \delta_t + e_{it}$

but using data for observations that were never unionized. That way you know, they no contamination effects should have occured. Once this is done, the treatment effect can be estimated using the following equation:

$y_{it} - \hat{\gamma_i} - \hat{\delta_t}= \theta * union_{it} + e_{it}$

Now that the individual and year fixed effects have been taken care of, the residual could be used to get a good estimation of the treatment effect.

Gardner (2021), calls this a 2-step DID appraoch. Stp 1. you get $\hat{\gamma_i}$ and $\hat{\delta_t}$ , and the second step, you estimate the treatment effect.

Borusyak, et al (2021) explains the process from a different perspective where they say:

$y_{it} - \hat{\gamma_i} - \hat{\delta_t} = \hat{\theta}_{it}$

And any treatment effect of interest could be obtained by getting averages for the treatment effects :

$TE_K = E(\hat{\theta}_{it}, w_{it})$

where $w_{it}$ are weights that depend on what is it you are trying to identify. Both authors have their own implementations. But I will use Borusyak's did_imputation, because is easier to use.

The data

So you should have a basic idea of what the estimator is trying to identify. It uses not yet treated observations to predict potential outcomes of non-treatment for all observations. Then uses the predicted nontreated outcomes to estimate treatment effects.

For the data, I decided to use the extract that is available from one of the example databases in Stata. The nslwork dataset. This sample only includes women, so the conclusions from here may not extrapolate to other populations. But should still be good for teaching purposes:

webuse nlswork, clear

This dataset has an indicator for whether or not an indivial declares to be unionized. However, this variable often has missing values. To get as much data as possible, I ll make some data modifications.

I will assume that if union is missing, the correct value is the last one observed.
Once an individual becomes unionized, union will be one for the rest of the time in the dataset.
If an individual never declare their union status (always missing), they will be dropped from the sample.
Otherwise, if union is missing in earlier periods, I will assume they were not unionized.

In additon to this, I also need to identify the "treatment" year. In absence of better data, I will simply assume that the treatment year is the first year when a persons status is unionized.

 bysort id (year):replace union = union[_n-1] if union==.
 bysort id (year):replace union = 1 if union[_n-1]==1
 bysort id (year):egen flag = sum(!missing(union))
 drop if flag==0
 replace union =0 if union==.
 bysort id: egen aux = min(year) if union==1
 bysort id:egen gvar = max(aux)

With this information, I have now all the ingredients to analyze the effect of union on wages.

Union Wage Effects

For the estimation of the impact of union on wages, I will use a very simplified model, using zero controls, other than the individual and time fixed effects. I will use the basic syntax in the did_imputation command.

ssc install did_imputation
ssc install event_plot
did_imputation ln_wage id year gvar, horizons(0/15) pretrend(10) autosample
event_plot, default_look

These seem as very promising results. Unions do have a positive impact on wages (about 15%), there is some anticipation, two or three periods before they change union-status. Perhaps due to wage negotiations or job change. But going beyond 3 or 4 periods in the past, I could argue the parallel trends asusmption holds.

The story also makes sense when you look into weeks work and hours of work!

ssc install addplot
did_imputation wks_work id year gvar, horizons(0/15) pretrend(10) autosample
event_plot, default_look 
addplot:,title("Wks work per year") legend(off)

did_imputation hours id year gvar, horizons(0/15) pretrend(10) autosample
event_plot, default_look 
addplot:,title("Hours work per week") legend(off)

Yes, there are some pre-trends, but the story seems reasonable. Higher wages, work longer shifts (full time rather than part time?) and more weeks per year.

But, how much can I believe these results.

Falsification test

As good exceptic, I decided to come up with a falsification test. The idea would be to create a setup where I know for sure there are no treatment effects. For the present example, I go with the following recipie:

Drop all observations who were ever unionized. (so no treatment effects should remain)
Assign a union status at random for everyone else left in the dataset. I assume that one cannot becomme uniozed during the first two years in the data.
Prepare all data (gvar, and union = 1 if treated in the past)
Repeat the analysis.

set seed 1
replace union = 0
replace union =1 if runiform()<0.05 & year >= 70  
bysort id (year):replace union = 1 if union[_n-1]==1
drop aux2 gvar2	
bysort id: egen aux2 = min(year) if union==1
bysort id:egen gvar2 = max(aux2)
did_imputation ln_wage id year gvar2, horizons(0/15) pretrend(10) autosample
event_plot, default_look

Because this falsification test is based on random assignment, I will use different seeds (feel free to change that) for the plots below. What is important, however, is that in this design there shouldnt be any treatment effects or anticipation, or pre-trends. Any deviations from that should be of concern.

Unfortunately, that is precisely what is being observed. In all the examples I show above, I find that there is some kind of treatment effect, with problems of anticipation.

The anticipation effect is just as big or bigger than the original data suggested. The wage premium effect is smaller (around 10%) and is often significant for the first few periods. However, based on the design, we shouldnt be observing this.

Conclusions

It is possible that this problem may be due to other time varying factors I'm failing to control. After all, the main purpose of this example was simply to see how this methodology can be used to analyze union wage gains, using a very simple dataset.

The analysis of union and wage gains deserve further exploring, however, as shown in this example, this new methodology shouldnt be applied blindly. And perhaps, at the very least, one should consider this falsification test as simple approach to verify one is estimating causal effects, and not effects driven by other unaccounted factors.

If you are interested in the full dofile that replicates this example. You can find it Here